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(54) A digital Interactive system for providing full interactivity with live programming events 



(57) The present invention relates to an Interactive 
digital system (1 ) enabling viewers full and active partic- 
ipation in experiencing a live broadcast event. Particu- 
lariy, the presentation of the live event is personalized 
for the viewer through the provision of options, including 
multiple video streams, associated with different cam- 
era angles, for example, and integrated audio and 
graphic segments. Further, information obtained from 
related Web sites can be integrated into the live pro- 
gram. 
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Description 

[0001] Interactive video and audio presentation 
systems are currently being introduced into the enter- 
tainment and educational industries. A prominent Inter- 
active technology that has been applied successfully in 
these industries is based on providing Interactivity in a 
one-way system through the provision of multiple time- 
synched parallel channels of Information. For example, 
U.S. patents nos. 4,264,925 and 4,264,924, provide 
both audio and video interactivity and disclose interac- 
tive television systems where switching among multiple 
broadcast or cable channels based on viewer selections 
provides an interactive capability. 
[0002] These systems have been enhanced to 
include memory functions using computer logic and 
memory, where selection of system responses played to 
the viewer is based on the processing and storage of 
subscriber responses, as described in U.S. patent no. 
4,507,680. 

[0003] The benefits of providing interactivity 
through the use of different audio responses is dis- 
closed in Freeman, U.S. patent nos. 4,847,698, 
4,847,699 and 4,847,700. These television systems 
provide a common video signal accompanied by several 
synchronized audio channels to provide content related 
user selectable responses. The audio signals produce 
different audio responses, and in some cases, these are 
syllable synched to a first audio script and to the video 
signal (such as to a person or character on a display), 
providing the perception that the person's or character's 
mouth movements match the spoken words. 
[0004] Interactivity is brought to the classroom in 
the Freeman U.S. patent no. 5,537,141. The distance 
learning system claimed in this application enhances 
the classroom educational experience through an inno- 
vative use of interactive technology over transmission 
independent media. When an instructor, either broad- 
cast live on video or displayed from videotape, asks a 
question, each and every student responds, preferably 
by entering a response on a remote handset, and each 
student immediately receives a distinct and substantive 
audio response to his or her unique selection. The indi- 
vidualization of audio response from the interactive pro- 
gram Is a major aspect of the invention. 
[0005] Individualization of audio is brought to the 
home based on the technology disdosed in Freeman 
U.S. patent no. 5,585,858. This system provides a pro- 
gram that can be watched on any conventional televi- 
sion set or multimedia computer as a normal program. 
But If the viewer has a special interactive program box 
connected to the television, he or she can experience a 
fully functional interactive program. Each interactive 
viewer enjoys personalized audio responses and video 
graphics overlayed on the screen. The Interactive pro- 
gram can be provided to television sets or to computers 
by cable, direct broadcast satellite, television broadcast 
or other transmission means, and can be analog or dig- 
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ital. 

[0006] However, what is needed is an interactive 
presentation system for providing tme video, audio and 
graphics Interactivity with digital programs broadcast 
5 live. Such a system must efficiently package all the dig- 
ital elements of the live interactive program at a central- 
ized control studio and allow viewers at home to receive 
personalized interactive programming. 

10 SUMMARY OF THE INVENTION 

[0007] In accordance with the invention, there is 
provided an interactive digital system allowing the 
viewer active participation in selecting digital video 

15 streams, associated with different camera angles, for 
example, and Integrated audio and/or graphics seg- 
ments. Further, Web pages from Internet Web sites can 
be integrated into the program. The Invention Is particu- 
larly suited for the environment of live events, such as 

20 the broadcast of live sporting events. The viewer can 
appear to direct the camera shots by instantly changing 
among various camera angles, choose player inter- 
views, or display associated statistical data on the team 
or players via graphics. In this manner, the system 

25 allows the individual subscriber to act as if he or she has 
control over how the program Is directed and presented 
on their personal television set. In addition to selecting 
different camera angles, various audio options, close- 
ups, slow motion, replays, graphics overlays, graphics 

30 or audio from Web sites, etc., are all possible. Further, 
games can be integrated with the live sports program- 
ming to Increase viewer interest. 
[0008] Thus, viewers can customize the content of 
programs. The interactive digital programming of the 

35 present invention Is particularly advantageous for view- 
ing live sporting events. Viewers are not limited to 
selecting from multiple camera angles, but may also call 
up player statistics on demand, listen to selected player 
interviews, etc. Cameras can be focused on different 

40 segments of an event. Further, video options could 
include video replay, slow motion effects, Isolation on a 
particular player or group, etc. Changes are seamless, 
thereby adding to the effect that the viewer is directing 
the television show just as a director now does from a 

45 control room. 

[0009] This "director" role by the viewer Is possible 
due to the interactive technology of the present inven- 
tion and also due to the digital compression and trans- 
mission scheme which allows for much greater 

50 information throughput over a given bandwidth, allowing 
viewers to choose from angles that are already availa- 
ble but presently cut by the director 
[0010] The digital interactive system is based upon 
branches which occur in the course of the full-motion 

55 video. Branches are real-time parallel paths that may be 
other full-motion video segments, graphics which are 
Integrated into the video, audio segments, and/or 
retrieved Web pages which are integrated into the live 
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event. 

[001 1] Sometimes, the interactive digital system will 
act upon the viewer's response immediately; other 
times, it will utilize ACTV's unique "profiling" concept to 
act upon the response later. This technology enables s 
the system to "remember" the viewer's responses and 
desires, and Integrate them into the video, audio, graph- 
ics and/or Web site information at a later point. For 
example, the viewer could specify at the beginning of a 
football game to isolate the offensive quarterback of a io 
particular team. Thus, whenever the team of choice is 
on offense, the video isolation of the quarterback is dis- 
played to the viewer automatically. Or, based on how a 
viewer has selected camera angles, replays, etc., over 
the past five minutes, the system acts to mimic these i5 
selections at later times during the program. The sys- 
tem of the present invention "learns" from the viewer 
how they want to view the game, and thus, continues 
viewer selection sequences made earlier. 
[0012] At the source, the present Invention com- 20 
prises a plurality of video cameras, each of the video 
cameras relaying a different predetermined view of an 
event. The video signals corresponding to the different 
cameras are fonwarded to a central control studio. Fur- 
ther, one or more audio signals or graphic statistical 25 
overlays can be collected and sent to a central control 
studio. After receiving the video, audio, and graphics 
signals at the central control studio, these signals are 
digitized and compressed In digital video and audio 
compressors. These signals are then combined with 30 
special data codes into a "digital package," and subse- 
quently, transmitted over a cable distribution system. 
The special data codes are the keys to unlocking the 
interactive potential of the program. 
[0013] The digital program signals are transmitted 35 
to a receive site by any suitable transmission means. 
Once received by a receive antenna, the digital program 
signals are passed along on a digital cable television 
distribution system to the viewer homes. Further, some 
other signals or commercials can be inserted at the 40 
local head end. The signals are received and processed 
in a digital cable box. Selections of the video, audio, 
graphics display and/or Web pages can be made as a 
function of Immediate viewer entries, or to inten-ogatory 
responses presented at the beginning or during the pro- 45 
gram, or based on a prestored viewer profile. Once a 
decision is made to switch from one video option to 
another video option, the digital switch is performed 
seamlessly 

[0014] As mentioned above, the program at prede- so 
termined times or immediately upon user entry can 
retrieve and branch to informative segments from Web 
sites. For example, a viewer watching a sporting event, 
through the system of the present invention, can receive 
a stream of Web pages which provide additional, spe- 55 
cific information relating to a favorite player, team or per- 
haps the remaining schedule for the sports team, as 
examples. In addition, users cast take advantage of the 
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two-way capabilities of the Internet to respond to polls 
or to link to additional sites. 

[0015] Another Internet-based application allows 
advertisers to speak more directly to consumers by 
directly sending Web pages to the consumer instead of 
merely displaying Web addresses in their commercials. 
The particular advertising information from Web sites 
can be targeted to viewers based on the viewer profile, 
stored either in the digital set top box or at the cable 
headend. Alternatively, Web site access can be initiated 
by the viewer by simply clicking on the remote during 
the commercial. Thus, viewers have the capability to 
individually select Web sites if they want more informa- 
tion from advertisers, for example. 
[0016] The video programming and corresponding 
internet pages can be viewed either on personal com- 
puters equipped with a television card on special digital 
cable boxes with stored interactive Internet application 
software providing Internet access, or on digital televi- 
sion sets, all of which would utilize the specialized 
TV/Internet software of the present invention. 
[0017] The present invention also has applications 
for other types of programming. For example, viewers 
can direct the scenes of a murder mystery. Switching 
from one scene to another can be done seamlessly 
without noticeable effect on the viewer. Further, the 
present invention can be used for any kind of live or pre- 
recorded event. For instance, a music concert or a polit- 
ical convention can be enhanced in the manner of the 
invention. 

[0018] Accordingly, a primary objective of this 
invention is providing an enhanced digital live program 
allowing the display to be tailored to the user's desires, 
choices or interests. 

[001 9] It is an object of this invention to personalize 
and enhance live sporting events for the viewer. 

DESCRIPTION OF THE DRAWINGS 

[0020] 

Figure 1 is a diagram of the network and equipment 
for providing live digital programming. 

Figure 2 is a block diagram of an interactive digital 
cable box allowing seamless switching between 
video signals. 

Figure 3 is a block diagram of an alternative dual- 
tuner interactive digital cable box allowing seamless 
switching between video signals. 

Figure 4 is a block diagram of another alternative 
interactive digital cable box allowing seamless 
switching between video signals. 

Figure 5 is a time diagram showing a representa- 
tion of trigger points and corresponding video. 
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audio and/or graphics segments, one or a combina- 
tion of which are selected for presentation to the 
subscriber immediately after the execution of the 
trigger point function. 

5 

Figure 6 is a block diagram of an alternative embod- 
iment of the interactive system including Internet 
access. 

Figure 7 is a block diagram of the two-way conflgu- io 
ration of the system. 

PESCRIPTION OF THE PR^F^R R^P EMBODI- 
MENTS 

15 

[0021] The present invention is an interactive digital 
system 1 for producing a powerful personalized pro- 
gram allowing the home viewer an expanded set of pro- 
gramming options. Digital TV streams are put into digital 
packages made up of video, audio, data codes and 20 
graphics, and are used to provide personalized 
responses to viewer selections. Such responses can be 
further enhanced by allowing access to Internet Web 
sites 170. In this manner, sports such as golf, football, 
baseball, basketball, etc. can now be watched with 25 
greater interest and Involvement. However, even further 
enhanced Interactivity Is possible with the present 
invention due to the provision of various profiling and 
memory features. 

[0022] As shown in Figure 1 , the present invention 30 
begins with the gathering of several possible video 
streams by way of cameras 100, strategically located at 
a sporting event 10, for example. Currently, many cam- 
eras 100 are employed at a sporting event 10. Super 
Bowl coverage, for example, typically encompasses 25 35 
to 30 cameras. These live video streams can be Inte- 
grated with recorded video streams which, for example, 
could include highlights from the current game or past 
games, player profiles, etc. To describe the components 
and operation of the present invention, the production 40 
and transmission of a live sporting event 10, I.e., foot- 
ball, is chosen to present the invention features. How- 
ever, other applications can be appreciated by the 
reader, including several disclosed below. 

45 

A. The Control Studio 

[0023] The interactive broadcast program Is pre- 
pared at the control studio 5 into digital packages. The 
control studio 5, as shown in Figure 1 , allows a producer so 
to create and Introduce Interactive elements during a 
live broadcast. In one preferred embodiment, the pro- 
ducer pre-records a set of Interrogatories or instructions 
for the user. These interrogatories may include such 
questions as the following: 55 

SELECT THE CAMERA ANGLES/OPTIONS 
THAT YOU PREFER: 



OPTIONS: END-ZONE 

FIFTY YARD LINE 
FOCUS ON THE QUARTERBACK 
FOCUS ON THE DEFENSE 
FOCUS ON THE CHEERLEADERS 
PROVIDE HIGHLIGHTS 

WHO IS YOUR FAVORITE TEAM IN THE GAME? 

OPTIONS: BLUE TEAM 
RED TEAM 

WHO IS YOUR FAVORITE PLAYER IN THE 
GAME? 

OPTIONS: RED RUNNER 

BLUE QUARTERBACK 
BLUE DEFENSIVE END 
RED QUARTERBACK 
ETC. 

ARE YOU INTERESTED IN RECEIVING STATIS- 
TICAL SUMMARIES? 

WOULD YOU ENJOY LISTENING TO PLAYER 
INTERVIEWS DURING THE GAME? 

[0024] Such interrogatories can be presented to the 
viewer at the beginning of the broadcast or scattered 
throughout the program. Interactive responses to such 
interrogatories would Include video, audio and graphics 
personalized to the particular viewer. The preparation of 
the graphics for presenting such Interrogatories occurs 
off-line at the control studio 5 using chyron or any graph- 
ics language. These interrogatories or instructions will 
ultimately be displayed to the home viewer, preferably in 
the form of graphics, to facilitate the interactive 
responses. The producer creates these graphic video 
slides of questions on a computer using the text editor 
and chyron. Associated with each question, the pro- 
ducer enters a number of possible viewer options. Then, 
the producer relates each possible user entry to one or 
more corresponding interactive responses. If the 
response is information from an Internet Web page, the 
producer will indicate the particular Uniform Resource 
Locator (URL) of the Web page. The producer sets a 
time stamp for when each particular question will 
appear during the program. As explained below in more 
detail, the viewer response to a query will be used to 
direct which video (and/or audio, graphics, or Web 
page) option will be provided to the viewer. In the 
present invention, the interactive response to the query 
can occur Immediately following the entry of the viewer 
entry or at some predetermined later time in the pro- 
gram using "trigger points," 500 as explained in detail 
below. 

[0025] With respect to the video segments, cam- 
eras 100 are preferably trained on different segments of 
the sporting event 10. As Is common with broadcasts of 
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a football game, for example, cameras 100 could be 
located in the endzone, press box, the field and at vari- 
ous other locations throughout the stadium. Further, 
various video options can be created Including video 
replay, slow motion, isolation on cheerleaders, particu- 
lar player or group of players. Instant replays are cre- 
ated by delaying the live feed for a certain number of 
seconds. These video streams are sent to a control stu- 
dio 5. The control studio 5 contains the necessary 
equipment for packaging the program for delivery to the 
viewers. The studio 5 contains a video switcher 105 
which receives the live signals from the cameras 100 by 
way of various input lines. Further, lines canning 
recorded video streams from one or more VCRs 110, 
computers or CD players feed into the video switcher 
105. The video switcher 105 also receives video Inputs 
from the control computer 135. Further, various graph- 
ics screens, depicting, for example, sports team or 
player statistics can be designed with the control per- 
sonal computer 135 and forwarded to the digital video 
switch 105. The producer, via the control PC 135, 
directs which video options to pass trough the video 
switcher 105. At the output of the video switcher 105, 
each of the different output video streams access a sep- 
arate encoder 125 and are all GENLOCKED, so that 
each video stream is synchronized with the other video 
streams. 

[0026] After encoding, the video streams are input 
into a video compressor 125. Preferably, the digital com- 
pression scheme is MPEG-2. Preferably, 64 Quadrature 
Amplitude Modulation (QAM) is used as the modulation 
scheme. In this digital embodiment, four channels of 
digitally-compressed video content would carry about 
27 Mbps using 6 MHz of bandwidth. Alternatively, if 256 
QAM is employed, seven-to-one video compression can 
be achieved with the MPEG-2 scheme. While MPEG-2 
is the prefen-ed compression scheme, the signals can 
be compressed according to any known standard 
including MPEG-1, JPEG, or other DCT coding 
scheme, wavelets, fractals or other transform or wave- 
form based technique. 

[0027] The control studio 5 also contains an audio 
switcher 115 which receives live audio signals from 
microphones or recorded audio from tape players 120, 
CDs, VCRs 1 10, etc.. The control computer 135 sends 
commands to the audio switcher 115 directing which 
audio options should pass through the switch 115. Fur- 
ther, in the audio switcher 1 15 the various audio signals 
can be aligned to match the various video signals In 
time. In addition, VCR audio output is received by the 
audio switcher 115. The present invention can accom- 
modate any number of audio signals as output from the 
audio switcher 115, as directed by the producer. The 
audio outputs are received by an digital audio 
encoder/compressor 130. The audio signals are then 
preferably sampled, encoded and compressed in the 
digital audio encoder/compressor 130. The encoding 
technique can be a waveform coding technique such as 
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PCM, ADPCM or DM. Alternatively, the signals can be 
encoded using synthesizer or vocoder techniques such 
as MUSICAM, Linear Predictive Coding (LPC), Adap- 
tive Predictive Coding (APC), and Sub-band coding. 
5 Generally, the transmission rate is about 256 kbps per 
audio for the stereo pair. 

[0028] The timing and control for integrating the var- 
ious multimedia elements is provided by the ACTV 
authoring language, a unique set of interactive data 

10 codes to facilitate the interactive process. The data 
codes are stored in memory in the control computer 135 
as part of the ACTV programming language. The codes 
comprise commands, or branch codes, for branching 
between interactive options, timing signals for control- 

15 ling the interactive program, data or text, commands for 
termination and Initiation or interactive program viewing, 
or triggers for executing macros. Preferably, these com- 
mands are output from the control computer 135 and 
multiplexed with the video streams in the MPEG-2 com- 

20 pressor 125, as shown In Figure 1. Interactive options 
that can be branched to based on the branch codes 
Include video segments, audio segments, graphics seg- 
ments and/or identified Web pages. 
[0029] There are several commands in the ACTV 

25 authoring language that function to provide the Interac- 
tive sports programming applications. These com- 
mands are explained below in Section D, which details 
programming applications. 

30 B. The Transmission System 

[0030] As shown in figure 1 , the digital interactive 
system 1 uses an interactive program delivery system 
with any transmission means Including satellite 15, 

35 cable 1 50, wire or television broadcast 1 75 to deliver the 
interactive program (hereinafter "composite interactive 
program") from the control studio 5 for distribution to 
subscribers In their homes. At the control studio 5, the 
signals from the digital multiplexer 140 are converted to 

40 RF and distributed to a microwave 175, cable 150 or 
satellite 15 network. Preferably, the digital interactive 
signal is fonwarded from the control studio 5 to a cable 
headend 150, and subsequently, sent to the homes via 
the cable network. 

45 [0031 ] The program is preferably the broadcast of a 
live event. For example, live sporting events with added 
interactive elements can be broadcast from the control 
studio 5. Such live interactive elements could be differ- 
ent camera angles 100, slow motion video, etc., as dis- 

50 cussed above, while also Incorporating prerecorded 
interactive segments such as highlights. Alternatively, 
the program can be produced off-line and stored in a 
program storage means at the control studio 5. 
[0032] In a satellite broadcast transmission, the dlg- 

55 Ital interactive signals are transmitted to uplink equip- 
ment where they may be multiplexed, upconverted, 
modulated, amplified and transmitted by satellite 15 to 
the receiver site 155 for distribution to the homes. 
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[0033] At the reception end, the composite digital 
interactive signals enter a receiver 1 55 where the sig- 
nals are demultiplexed, downconverted, demodulated 
and then passed to a cable distribution system that 
directs the signals to the homes. Although a cable distri- 5 
bution system 150 is the preferred transmission media 
to the homes, the digital signals may also be distributed 
by any conventionally known technique including satel- 
lite 15 to digital satellite receivers 155 at the home, 
fiberoptics, low or high power broadcast television 175, to 
telephone lines, cellular networks, and similar technol- 
ogy can be used interchangeably with this program 
delivery system. 

C. The Interactive Digital Box 15 

[0034] The interactive digital box 25 is shown sche- 
matically in Figure 2. Preferably, the interactive digital 
box is a specially adapted digital cable box 25. The con- 
troller 260 determines what video, audio, graphics 20 
and/or Web pages to display based upon the interactive 
commands which it receives. Based upon the com- 
mands, it plays the appropriate video, audio, graphics or 
Web page options. The graphics can either be created 
and sent from the control studio 5 or the graphical 25 
images can be created at the interactive digital box 25 
based on instructions preferably in the interactive com- 
mands. The interactive digital box 25 connects to a tele- 
vision 165 or other display monitor. Further, the 
interactive digital box 25 can be connected to a digital 30 
television 195, in which case an RF modulator 245 is 
not necessary. Each downstream transmission reaches 
the subscriber's house, shown in Figure 2, preferably 
through a tap and drop cable. 

[0035] The user interacts with the program through 35 
the input device 20. Preferably, the input device 20 is a 
typical television remote. The user interface 270 may be 
an infrared, wireless, or wired receiver that receives 
information from the input device 20. 
[0036] Regardless of the type of input device 20, 40 
user inputs can be utilized by the present invention 
immediately, or at a later time, to result in personalized 
graphics, video and/or audio presentation. For example, 
the present invention utilizes "trigger points," 500 as 
described below, to enable subsequent branches 45 
among multimedia segments during the show. 
[0037] Certain commands are sent from the control 
studio 5 as part of the digital interactive programming to 
facilitate the collection of user entries. These com- 
mands are extracted at the digital demultiplexer 21 0 and so 
sent to the controller 260 which performs the appropri- 
ate action based on the commands. Some of these 
commands are explained below. 

a, BEGIN INPUT EXTENDED 55 

The Begin Input command starts an input 
period during which the user may press one or 
more buttons to select his or her choice(s). The 
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entry format of this command is set forth as follows: 

BEGIN INPUT {{IN} {VALID KEYS} {DIFFER- 
ENT} {FEEDBACK} 

where: 

N The maximum number of keys 

that can be pressed. 

KEYS The valid keys that can be 
pressed. 

DIFFERENT Requires each key pressed to be 
different. 

FEEDBACK The feedback type provided to 
the viewer. 

b. BEGIN VIDEO CHOICE EXTENDED 

The Begin Video Choice begins an input period 
for disco mode. During disco mode, the video 
switches dynamically each time the viewer makes a 
selection, 

BEGIN_VIDEO_ CHOICE MODE [KEYS] 
{FEEDBACK [AUDIO]} 

The disco mode allows the viewer to change 
channels at will, while the OneShot mode allows 
only one change of channel. 

MODE DISCO/ONESHOT 
KEYS The valid keys that can be pressed. 
FEEDBACK The feedback supplied to the user for 
the key(s) which are pressed. 

The mode Disco allows the viewer to change 
channels at will, while the OneShot mode allows 
only one change of channel. 

c. BEGIN AUDIO CHOICE EXTENDED 

The Begin Audio Choice begins an input period 
for disco mode. 

BEGIN_AUDIO_CHOICE MODE [KEYS] 
{FEEDBACK [AUDIO]} 

MODE DISCO/ONESHOT 
KEYS The valid keys that can be 
pressed. 

FEEDBACK The feedback supplied to the user 
for the key(s) which are pressed. 

d. MAP 

The Map command is used to map video or 
audio tracks to keys, for use in connection with the 
Begin Audio Choice and Begin Video Choice com- 
mands. If this command is omitted. Key 1 will map 
to Track or Channel 1 , Key 2 to Track or Channel 2 
etc. This command allows mapping any key to any 
channel. 
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MAP KEYS [TRACKS [T1,...TN/T1-TN] 
CHANNELS [C1,.,.CN/C1-CN]] 

The map statement maps audio tracks or video 
channels to keys, to enable audio or video choice 
commands to effect changes to tracks other than 
the default tracks, which are that key 1 maps to 
track 1, key 2 to track 2, etc.. All the choice state- 
ments after a map statement will cause the tracks, 
video tracks or channels to be changed to those 
specified in the map statement. The acceptable 
numbers for the map command are from 1 to 8, for 
keys, video channels and audio channels. 

[0038] Other commands include those which allow 
for the following applications: 

(1) viewer profiling, to enable the set top box 25 to 
"remember" viewer preferences; (2) uploading 
viewer responses to a central location; (3) down- 
loading of text and graphics, for display using the 
graphics chip of the set top box 25; (4) the ability of 
the viewer to prepare his own video, based upon his 
selections of camera shots 100 and audio, which 
can be stored and replayed for the viewer. 

[0039] The interactive digital box 25 of the present 
invention enables seamless flicker-free transparent 
switching between the digital video signals. "Seamless" 
means that the switch from one video signal to another 
Is user Imperceptible. Because the video signals are 
running off the same clock, the interactive digital box 25 
is capable of providing a seamless digital switch from 
one video signal to another signal. The program dock 
reference necessary for the box to make this seamless 
switch is preferably embedded In the signal header. 
[0040] As shown In Figure 2, a CPU 260 Is con- 
nected to an RF demodulator 200 and digital demulti- 
plexer 210. The CPU 260 directs demodulation and 
demultiplexing of the proper channel and data stream to 
obtain the correct video signal. Seamless switching can 
occur with MP EG -2 compressed signals since there are 
points within the frame wherein seamless switching can 
occur. Preferably, switches occur at an "1" frame, assum- 
ing the use of MPEG-2 compression. The selected 
video signal is determined either by examination of the 
user's input from user Interface 270 and/or any other 
information or criteria (such as personal profile informa- 
tion) stored in RAM/ROM 265. For example, the 
RAM/ROM 265 could store commands provided within 
the video signals as discussed In U.S. patent no. 
4,602,279, and incorporated herein by reference. 
[0041] The RF demodulator 200 demodulates data 
from the broadcast channel directed by the controller 
260. After the data stream is demodulated. It passes 
through a fonward en'or correction circuit 205 into a dig- 
ital demultiplexer 210. The demultiplexer 210 is control- 
led by the controller 260 to provide a specific video 
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signal out of a number of video signals which may be 
located within the data stream on the demodulated 
broadcast channel. The demultiplexed video signal Is 
then decompressed and decoded by decompres- 

5 sor/decoder 215. The video signal Is synchronized by a 
sync add circuit 220 and a sync generator 225. The 
video signal is then buffered by a video frame buffer 
230. The buffered video signal is modulated by a modu- 
lator 245 into a NTSC compatible signal. Such a modu- 

10 lator is not necessary if the selected signal is sent to a 
digital television 195. 

[0042] By using a video frame buffer 230 and delay- 
ing the viewing of a given signal, enough time is allowed 
for the decompressor/decoder 215 to lock onto, decom- 

15 press, convert to analog, and wait for the resultant verti- 
cal interval of a second video signal. For example, 
assume video signal A Is cun'ently being processed and 
transferred through the circuit shown In Figure 2 and 
displayed. Based upon a user selection, the controller 

20 260 directs the digital demultiplexer 210 and RF demod- 
ulator 200 to switch to another video signal, video signal 
B. To accomplish this, the analog video from the first 
digital video signal, video signal A, complete with video 
sync, is fed into video frame buffer 230. This buffer 230 

25 can hold the full video picture for "n" number of frames 
after which the signal Is output to the display In effect, a 
delayed video signal A is viewed "n" number of frames 
after the signal has been received. When the user 
selects a different video path by means of pressing a 

30 button on a keypad or entry by other means, the control- 
ler 260 instructs the digital demultiplexer 210 to stop 
decoding signal A and lock onto signal B to begin 
decoding signal B Instead of signal A. 
[0043] While this Is happening, even though the 

35 decompressor/decoder 2 1 5 is no longer decompressing 
video signal A, the display Is still showing video signal A 
because it Is being read from the buffer 230. As soon as 
decompressing and decoding occurs, the controller 260 
looks for the next vertical blanking interval (VBI) and 

40 instructs the video frame buffer 230 to switch to Its input, 
rather than its buffered output at the occurrence of the 
VBI. 

[0044] Since the RF demodulator 200, fonward error 
corrector 205, digital demultiplexer 210, and decom- 

45 pressor/decoder 215 require a certain time period to 
decompress and decode the video signal B frame from 
Its data stream, the size of the buffer 230 has to be large 
enough so that this processing can take place without 
interruption during the switching of the video signals. If 

50 desired, the system may continue to use the buffer in 
anticipation of a future switch. By using the controller 
260 to manipulate the fill and empty rate of the buffer 
230, the buffer 230 may be rapidly filled with video sig- 
nal B frames and then after a period of time will be reset 

55 and ready to make another switch to another video In 
the same manner. The buffer 230 may also be reset by 
skipping frames or providing a delay between sequen- 
tial frame outputs for a short time In order to fill the buffer 
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230. If a delay is used to maintain video signal or frame 
output while the buffer 230 is being filled, a slight distor- 
tion may occur for a brief amount of time. 
[0045] Because a first video signal is always dis- 
played as the output of the buffer 230 after the delay, the 
buffered video masks the acquisition and decoding of a 
second video signal. As long as the buffer 230 is large 
enough to keep the first video running while the second 
video is being decompressed and decoded, a seamless 
switch will occur 

[0046] While the digital interactive box 25 of Figure 
2 provides video interactivity, audio and/or graphics 
interactivity is also provided. For examples if, based on 
the viewer profile or viewer response to query, it Is 
determined that the viewer's primary language is Span- 
ish, then that viewer could obtain Spanish commentary 
to the football, soccer, etc. game. Alternatively, if a 
viewer has a favorite athlete, the audio can switch to an 
Interview with the athlete during a segment of the 
broadcast. Multiple digital audio options forming a set of 
suitable responses to an interogatory message can be 
sent as part of the composite digital signal. As set forth 
in U.S. patent no. 5,585,858, herein incorporated by ref- 
erence, there are a number of different ways to effec- 
tively fonward the necessary audio options for a given 
live event to the digital interactive box 25. With the 
present Invention, it makes no difference how the audio 
options reach the digital interactive box 25, as long as 
they are available for selection and play at the appropri- 
ate times. 

[0047] In Figure 2, the digital demultiplexer 210 
extracts the digital audio signal (s) and fonwards them to 
the audio switch 250. Additional audio options axe avail- 
able from the digital audio memory 255. At certain times 
during the program, the data codes will identify the 
selection of a particular audio option corresponding to 
previous user inputs. The controller 260 calls the appro- 
priate audio options from internal memory 255 or 
directs the audio switch 250 to select a predetermined 
audio segment received as part of the received digital 
signal for passage to the RF modulator 245 for play to 
the subscriber. At the end of the audio segment time 
period as indicated by the data codes, the controller 260 
instructs the audio switch 250 to again pick up standard 
audio. 

[0048] The digital demultiplexer 210 sends the 
extracted graphics data or ACTV data codes to the con- 
troller 260. The controller 260 interprets the extracted 
data as either control data, including instructions for 
switching between video signals, audio signals, or 
graphics data for on-screen display. If the data is on- 
screen display data, the data is preferably prefixed by a 
command designating the data as on-screen display 
data, as opposed to control data. Further, the controller 
260 also examines the control data for the occurrence of 
a header code designating the onset of a trigger point 
500 in the program, explained below. 
[0049] Figure 3 shows an alternate, dual tuner 
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embodiment for seamless switching between separate 
video signals. This embodiment presumes that two 6 
MHz channels are used, each of which comprises com- 
pressed digital video and audio streams. In this embod- 

5 iment, the microprocessor 260 controls the selection of 
the RF channel that is demodulated by RF demodula- 
tors 200A, 200B. The demodulated data streams enter 
the fonA^ard error correctors 205A, 2058. At the output of 
the fonward error correctors 205A, 205B the data 

10 streams are transmitted to the input of the digital demul- 
tiplexers 21 OA, 21 OB. 

[0050] As with the RF demodulators 200A, 200B, 
the digital demultiplexers 21 OA, 2108 are controlled by 
the microprocessor 260. This configuration allows the 

15 microprocessor 260 to independently select two differ- 
ent Individual time-multiplexed video signals on different 
channels and data streams. If all the video signals of an 
interactive program were contained on a single channel 
or data stream. It would only be necessary to have a sln- 

20 gle RF demodulator 200, fonward error corrector 205, 
and digital demultiplexer 210 serially connected and 
feeding into the two digital video buffers 230A, 2308. 
[0051] Two data streams are provided from the dig- 
ital demultiplexers 21 OA, 21 OB. One data stream carries 

25 video information pertaining to the video signal the user 
is currently viewing. The second data stream carries the 
video signal selected based on the user's previous 
and/or cun-ent interactive selections from the user inter- 
face 270, as determined by the microprocessor 260. 

30 [0052] The digital information on each of the two 
streams is buffered in digital video buffers 230A, 230B. 
The buffered signals are then decompressed and con- 
verted into analog signals by decompressors/decoders 
21 5A, 2158 which include digital to analog converters. 

35 The decompressors 215A, 2158 are preferably MPEG- 
2 decoders. 

[0053] A local sync generator 225 is connected to 
sync add 220A, 2208 and frame sync 380A, 3808 cir- 
cuits. Because both streams are synchronized based 
40 on signals from the same local sync generator 225, 
each stream becomes synchronized to the other. In par- 
ticular, the signals on each stream are frame synchro- 
nized. 

[0054] A vertical blanking interval (VBI) switch 335 
45 is connected to the microprocessor 260 so that the Input 
may be switched during the vertical blanking interval of 
the current stream, resulting in a seamless switch to the 
viewer. 

[0055] The embodiment of Figure 3 operates as fol- 
50 lows. Based on user responses and control codes, it is 
assumed that the microprocessor 260 determines that a 
switch from video signal A to video signal C should be 
performed. The RF demodulator 200A and digital 
demultiplexer 21 OA are processing the currently viewed 
55 video signal, video signal A, which is progressing 
through the upper branch components. A command is 
issued from the microprocessor 260 to the RF demodu- 
lator 200A, 2008 commanding a switch to the channel 
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and data stream on which video signal C Is located. The 
microprocessor 260 also instructs the digital demulti- 
plexer 21 OB to provide video signal C from the received 
data stream to digital video buffer 2306. 
[0056] At this point, the upper RF demodulator 
200A and digital demultiplexer 21 OA are still independ- 
ently receiving and processing video signal A, which 
continues through the upper branch of the circuit. 
[0057] At a certain point, the digital decompres- 
sor/decoder 21 SB in the lower branch will begin filling 
up with video signal C frames. After video signal C is 
decompressed and decoded, it is converted Into analog. 
A local sync generator 225 inserts both local sync and 
frame sync to video signal C via sync add circuit 220B 
and frame sync circuit 380 B In order to synchronize it 
with the currently displayed video signal A, which Is still 
being provided from the upper digital video buffer 230A. 
At the appropriate switch point, triggered by program- 
ming codes supplied with each video signal A and C, 
the microprocessor 260 directs the VBI switch 335 to 
switch in the vertical blanking interval from video A to 
video C, at which time video C will then seamlessly 
appear on the computer screen. 
[0058] Digital video buffers 230A, 230B may be 
used In the circuit of Figure 3, but are optional. However, 
in an alternative embodiment the buffers would be 
required to provide a seamless switch if the Figure 3 cir- 
cuit was modified to Incorporate a single RF demodula- 
tor 200, single fonward error corrector 205, and single 
digital demultiplexer 210, each with a single Input and 
single output In this alternative embodiment, the circuit 
cannot independently receive and demultiplex two data 
streams on different frequency channels. One buffer 
230A Is used to store previously received video signals, 
while the other buffer 230B quickly passes through the 
selected video signals. 

[0059] Based on the same assumptions above, 
video signal A is progressing through the upper branch 
of the circuit and it Is desired to switch to video signal C. 
However, in this alternative embodiment, the digital 
video buffer 230A Is providing maximum buffering to 
video signal A, 

[0060] Because It Is desired to switch to video sig- 
nal C, the microprocessor 260 directs the alternative cir- 
cuit (containing a single RF receiver 200, single fonvard 
error corrector 205 and single digital demultiplexer 210 
connected in serial), to receive and demultiplex the data 
stream on which video signal C is located, which may 
be different than that of video signal A. When video sig- 
nal C is demultiplexed, the microprocessor 260 directs 
the digital video buffer 230 to provide minimum buffering 
of video signal C so that decompressor/decoder 215 
may quickly decompress and decode the digital signals. 
After decompression and decoding, video signal C Is 
synchronized with video signal A. At this time, video sig- 
nal A Is read for display from digital video buffer 230A. 
The upper digital video buffer 230A must be large 
enough to provide video frames for output during the 



time it takes the RF demodulator 200 and digital demul- 
tiplexer 210 to switch to video signal C and the time 
required for decompression, decoding, and synchroni- 
zation of video signal C. 

5 [0061] When video signal C is synchronized with 
video signal A, the microprocessor 260 directs VBI 
switch 335 to switch from video signal A to video signal 
C In the vertical blanking interval of video signal A, 
thereby providing a seamless and flicker-free switch. 

10 [0062] At this time, digital video buffer 230 will begin 
to utilize maximum buffering by altering its fill/empty rate 
as described above with respect to the Figure 3 embod- 
iment. When adequate buffering is achieved, a switch to 
another video signal may be performed in the same 

75 manner as described above. 

[0063] Another preferred embodiment is shown in 
Figure 4. This embodiment also Includes an RF demod- 
ulator 200, a fonward error corrector 205, and a digital 
demultiplexer 210. However, the circuitry differs along 

20 the rest of the chain to the television set or monitor. In 
this embodiment, a memory 475 is incorporated and 
connected to the output of the demultiplexer for storing 
the compressed composite digital video signal. The 
decompressor/decoder 215 is inserted at the output of 

25 the compressed memory. The decompressor/decoder 
21 5 decompresses the digital signal, converts the signal 
to analog and fonwards the analog signal to the RF 
encode 245 for transmission to the monitor. Once the 
composite compressed digital video signal is fed into 

30 the compressed memory 475, the microprocessor 260 
directs a pointer to be placed somewhere along the 
compressed digital video signal. Based on the place- 
ment of the pointer, different frames and different seg- 
ments of the composite digital video signal will be read 

35 from memory 475 for decompression and decoding. 
[0064] The different video signals are distinguished 
from one another because they are labeled, preferably 
by headers. Assuming that video signal A has been 
selected for play on the monitor, the compressed digital 

40 memory 475 fills up with A frames. Assuming a switch to 
video signal C is desired, the microprocessor 260 
directs the RF demodulator 200 and digital demulti- 
plexer 210 to begin filling the compressed memory 475 
with video 0 frames. The decoder 215 pointer begins to 

45 move down. As soon as a sufficient number of C frames 
have entered the compressed memory 475, the pointer 
will then jump to the beginning of the C frames. The C 
frames are then output Into the decompressor/decoder 
21 5 where the digital frames are converted into an ana- 

50 log signal. 

[0065] The digital video is multiplexed In a series of 
easily identifiable packets. These packets may contain 
full compressed frames of video (I frames) or may 
include only the differences between full frames (B 

55 frames or P frames). 

[0066] To be able to reconstruct the full video 
images, the decompressor/decoder 215 needs to have 
a minimum number of I, P and B frames. The decoder 
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215 needs only one I frame to decode an image. Con- 
versely, two prior Anchor frames ("I's" and "P's") are 
necessary to decode B frames. In order to decode P 
frames, the decoder 215 only needs one Prior Anchor 
frame. When the microprocessor instructs the digital 5 
demultiplexer 210 to start sending packets from a differ- 
ent data stream there is no way to be certain that the 
next packet will be an I packet needed for decoding the 
second video stream. To avoid a breakup of the video 
images, which would occur if the decompres- 10 
sor/decoder 215 suddenly started receiving packets 
unrelated to the stream it was decoding, the microproc- 
essor 260 starts to fill up the memory 475 with video 
signal C packets until it is determined that a full 
sequence of I, B and P frames are available. The 15 
decoder 215 should receive the last bit of the last B 
frame in a given, GOP (Group of Pictures) before the 
switch, in order to prevent glitches when decoding. Fur- 
thermore, the last B frame of the GOP must only be 
backward predicted, not fonA^ard predicted or bidirec- 20 
tlonal predicted. As soon as the valid sequence is in 
memory 475 the microprocessor 260 moves the mem- 
ory read pointer to the start of a valid sequence of C 
video signal packets so that the decompressor/decoder 
215 can successfully decode the C signals. This results 25 
In a seamless switch from video signal A to video signal 
C. 

[0067] This embodiment requires a data channel for 
enabling a synchronous switch between a first video 
stream and a second video stream. This data channel 30 
comprises the ACTV codes which link together the dif- 
ferent program elements and information segments on 
the different video signals. In addition, the data channel 
also comprises synchronization pulses and a time code 
to signify to the pointer the proper time to skip from a 35 
memory location representing one video signal to a 
memory location representing another video signal in 
order to enable a seamless switch. 
[0068] The microprocessor 260 reads the data sig- 
nal from the digital demultiplexer 210 and communi- 40 
cates pertinent data to the sync add circuit 220, which is 
connected to sync generator 225. The microprocessor 
260 is then able to synchronously communicate with the 
memory 475. 

[0069] The time code sent will identify the timing for 45 
one picture, as well as for multiple pictures, and will lock 
the different pictures together. This is done through the 
use of similar clocks at both the transmission end and 
the receiver. A time code is used in order to keep the 
two clocks at both the transmission and receive end so 
synchronously connected to one another. Once the 
clocks at both ends are working synchronously, each of 
the multiplexed video streams must be synchronized to 
the clocks. In order to synchronize the multiplexed video 
stream to the clocks, each of the individual channels 55 
must be referenced to a common reference point and 
must be identified. 

[0070] In the preferred embodiment, a packet 



header would be incorporated into the transport layer of 
the MPEG signal to identify the various channels. The 
packet header will also include information as to where 
to insert the vertical blanking interval. In MPEG, the ver- 
tical blanking interval is not transmitted from the head- 
end. Therefore, the vertical blanking interval must be 
generated locally. The packet header eye will identify at 
what time the vertical blanking interval is in existence in 
order to effectuate a seamless switch between analog 
pictures. 

[0071] In summary, the combination of clock and 
the information embedded in either the transport layer 
of MPEG or in a separate packet on a separate data 
channel effectuates the linking between each video sig- 
nal and a corresponding time point. The data channel 
also includes information designating when all the vari- 
ous video signals will be in synchronism with one 
another. It is at these points that the microprocessor 260 
may direct the pointer to skip from one location to 
another location, at a time (such as during the VBI) 
when a seamless switch will result. 

D. TnggenEQiQ^ 

[0072] Interactivity is further enhanced in the digital 
interactive embodiments through the application of trig- 
ger points 500 scattered at various predetermined times 
throughout the program, a timeline representation of 
which is shown in Figure 5. The trigger points 500 cor- 
respond to times when interactive events are scheduled 
to take place during the live sporting event 10. These 
interactive events could be the selection and playing of 
video, audio segments, the display of graphics or dis- 
play of Web pages accessed from Internet Web sites 
170. For example, when a viewer's favorite baseball 
player is at bat, graphics showing past or current per- 
formance statistics of the player can be overlaid on the 
screen while excerpts from an interview with the player 
can be played for the viewer. While the choice of partic- 
ular video, audio or graphics is still dependent on viewer 
selections, the viewer selections in response to dis- 
played graphical inten-ogatory messages are preferably 
made during a period at the onset of the program or 
when a viewer first tunes into the program. Alternatively, 
interrogatories are not necessary if the switches are 
based on the viewer profile stored in memory 265. 
These viewer selections are then utilized as inputs to 
macros called up at later times during the program by 
the controller 260 upon the occurrence of the trigger 
points 500, identified to the interactive computer by 
unique codes embedded in the video signal. 
[0073] The trigger points 500 correspond to the 
times when the conventional program content can be 
altered and personalized for the viewers. The program- 
mer can place the trigger points 500 at any time 
throughout the program. Since the trigger points 500 
are unknown to the subscriber, the subscriber does not 
know when they will receive a personalized message. In 
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other words, an interactive response can either immedi- 
ately follow a corresponding user selection made to an 
inten'ogatory message or occur at a later time corre- 
sponding to a trigger point 500, or any combination of 
the two. Of course, timing of the interactive events 5 
should correspond to suitable times In the program 
where branching to Interactive elements is sensible and 
does not clash with the program content of the conven- 
tional video still displayed on the television 165 or other 
display monitor. 

[0074] At the onset of a trigger point 500, the con- 
troller 260 will select one of several possible audio (or 
video or graphic display) responses for presentation to 
the subscriber. As mentioned above and shown In figure 
5, some of the responses may comprise a branch to is 
either a video segment, graphics and/or audio seg- 
ments. 

[0075] In combination with the use of trigger points 
500, the present invention allows for the viewer to select 
certain options at the onset of the program to suit the 20 
viewers' preferences. For example, if the program 
broadcast is a live sports event 1 0, at an early trigger 
point 500, the viewer could be queried as to whether the 
viewer would prefer to receive audio in English, Span- 
ish, French, or pertiaps hear the local announcer 25 
instead of the network announcer. Upon the viewer 
selection, the CPU 260 directs a branch to the appropri- 
ate interactive segment. 

[0076] Each trigger point 500 Is identified preferably 
through the broadcast of ACTV codes sent as part of 30 
the composite interactive program signal. The codes 
preferably include, at a minimum, the following informa- 
tion: (1) header identifying the occurrence of a trigger 
point 500; (2) function ID (e.g., selection of audio or 
graphics responses, etc.); and (3) con-esponding Inter- 35 
rogatory message(s) or particular viewer characteristic 
or habit based on viewer profile. The first bit sequence 
simply Identifies to the controller that a trigger point 500 
Is about to occur. The function ID designates the macro 
or other set of executable instructions for the controller 40 
260 to read and interpret to obtain the desired result, 
e.g., a selected video and/or audio response. 
[0077] Upon extraction of the codes by the data 
decoder, the controller 260 reads and interprets the 
codes and calls from memory 265 a particular user 45 
selectlon(s) designated by the trigger point 500 codes. 
The user selections correspond to subscriber answers 
to a series of interrogatory messages preferably pre- 
sented at the beginning of the program. After obtaining 
the appropriate user selectlon(s), the controller 260 so 
reads and performs the executable instructions using 
the user selectlon(s) as input(s) In the macro algorithm. 
The result of the algorithm is either a selected video 
stream, audio and/or selected graphics response. The 
video/audio response can be called from memory 265 if 55 
It is prestored, called from external data storage, or the 
controller 260 can command the switch to branch to the 
particular video audio stream if the response Is broad- 



cast concurrently with the trigger point 500. After the 
selected video/audio response is played to the sub- 
scriber, the switch branches back to the standard pro- 
gram, shown at time tg in Figure 5. 
[0078] As mentioned above, a series of interroga- 
tory messages are preferably presented when the sub- 
scriber begins watching the Interactive program. These 
Interrogatory messages can be presented in any one of 
three ways. First, the interrogatory messages can be 
presented as graphics displays overiaid by the Interac- 
tive computer workstation onto a video signal, wherein 
the graphics data is sent In the vertical blanking Interval 
of the composite interactive signal, or alternatively 
stored on the hard disk or external storage. Second, the 
interrogatory messages are presented as graphics dis- 
plays as discussed above, except the graphics data 
comes from local storage, external data storage (e.g., 
CD ROM, cartridge, etc.), or a combination of data in 
the VBt and data called from either local or external data 
storage. Third, graphics data can be presented In the 
form of user templates stared at the interactive compu- 
ter workstation. 

[0079] User selections corresponding to answers to 
the n successive Interrogatory messages are received 
by the remote interface 270 at the beginning of the 
show, stored In memory 265 and used throughout the 
show at the appropriate trigger points 500 to subtlety 
change program content as the show progresses. Pref- 
erably, each interrogatory has a set of possible answers. 
Next to each possible answer will be some identifier cor- 
responding to a label on a key on the user Interface. The 
subscriber depresses the key con^esponding to their 
answer selection. This selection is decoded by the 
remote Interface 270 and controller 260, stored In mem- 
ory 265, preferably RAM, and used later as required by 
an algorithm designated at a trigger point 500. 

E. Internet 

[0080] In addition to the central studio serving as a 
source of interactive option responses, the Internet can 
be used as a source of personalized information for 
interactive responses. 

[0081] As discussed above, the video programming 
is preferably created at a centralized location, I.e., the 
control studio 5 as shown in Figure 1 , for distribution to 
subscribers in their homes. Referring to Figures 1 and 
6, in a prefen-ed method, the operator at the control stu- 
dio 5 must designate certain Web pages to correspond 
to one of the program options, such as audio and/or 
graphics options, using control PC 135 and URL 
encoder 600. Instead of encoding the actual content of 
the options at the control studio 5, as with the audio, 
Web address identifiers, i.e., Uniform Resource Loca- 
tors (URLs) are encoded and sent as part of the data 
codes from the control PC 135 to the digital multiplexer 
140. After multiplexing, the URLs are sent as part of the 
program signal 615, as described above. Preferably, the 
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URLs, like the various audio and graphics options, have 
associated time stamps which indicate to the remote 
digital set top boxes 25 when, during the video program, 
to display the particular Web pages addressed by the 
URLs, the selection and display of which is preferably 5 
made as a function of viewer responses or viewer pro- 
file. 

[0082] Preferably, each digital set top box 25 has an 
Internet connection 160 created concurrently with the 
cable connection. The Internet connection 160 can be 10 
via high-speed line, RF, conventional modem. The dig- 
ital set top box 25 has Internet access 160 via any of the 
current ASCII software mechanisms. In a prefered 
embodiment, In the interactive digital set top box 25, the 
digital demultiplexer 210 extracts the URLs along with 15 
the other data codes. In an alternative embodiment, a 
local URL decoder 605 at the user site extracts the 
URLs. 

[0083] In a preferred embodiment, a JAVA enabled 
browser as well as specialized software for performing 20 
part of the method of the present invention are installed 
on the Interactive digital set top box 25. The JAVA ena- 
bled browser allows the interactive digital set top box 25 
to retrieve the Web pages and is prefen'ed software, 
since It is platform Independent, and thus, enables effi- 25 
cient and flexible transfer of programs, images, etc., 
over the Internet. The specialized software acts as an 
Interface between the video programming and the Inter- 
net functions of the present invention. The processor 
and software Interprets these URLs and directs the 30 
JAVA enabled browser to retrieve the particular relevant 
Web pages, and synchronizes the retrieved Web pages 
to the video content for display on the television monitor 
165 at the appropriate times. 

[0084] In the present invention, the viewer also has 35 
the capability to link to a channel website at will. For 
example, if a viewer Is Interested in purchasing a prod- 
uct described In an advertisement, by merely clicking on 
a button on their remote 20, the producer's Website 
could be accessed by Internet connection 160 and dis- 40 
played to the viewer. The viewer could then either obtain 
more Information about the product or order the prod- 
uct, if desired. As described above, this application is 
possible by sending the URL associated with the pro- 
ducer's Website to the digital cable boxes 25 as part of 45 
the interactive program. Upon selection by the viewer, 
the web browser, located either in the digital set-top box 
25 or externally in a connected PC 610, can retrieve the 
Web pages. The specialized software then synchro- 
nizes the Web pages for video display. so 

F. Memory 

[0085] The interactive digital set top box 25 of the 
present invention also has the advantage of remember- 55 
ing subscriber responses and using these responses in 
choosing a video/audio response, and/or graphics inter- 
rogatory message, to present to the student. Memory 



branching is a technique of the present invention where 
the algorithm assembles video/audio responses and 
graphics interrogatory messages according to the cur- 
rent and previous user Inputs. Memory branching is 
accomplished by linking video/audio streams and/or 
successive graphics interrogatory messages together In 
a logical relationship. In this scheme, the Interactive dig- 
ital set top box 25 contains logic (preferably, In the soft- 
ware algorithm) and memory 265 to store previous 
subscriber selections and to process these previous 
responses In the algorithm to control future video/audio 
stream selection, as well as future graphics message 
selection. 

G. Dloital Viewer Profiles 

[0086] In a prefen'ed embodiment, the interactive 
digital cable box 25 can have a "viewer profile" stored In 
its memory 265. Alternatively, the accumulated profile of 
viewer characteristics and/or habits can be stored at the 
control studio 5 or cable headend. If the profile statistics 
are accumulated at some central location, they can also 
be sent to each viewer's home. On the other hand. If 
accumulated in memory 265 at each of the set top ter- 
minals 25, the data could be sent to the central location 
for storage and dissemination by way of the digital back 
channel. Decisions regarding personalized advertising 
and viewing can then be made for a viewer or class of 
viewers based on the accumulated parameters. 
[0087] The "viewer profile" preferably contains char- 
acteristics of the particular viewer at that subscriber 
location, such as sex, hobbies, interests, etc. This 
viewer profile is created by having the viewer respond to 
a series of questions. Alternatively, the viewer profiles 
could be created at a control studio 5 and sent to the 
Interactive digital cable box 25. This information is then 
used by the cable box software to create a compendium 
of the viewer's interests and preferences -- i.e., a user 
profile. The stored user profile would be used in place of 
the question/answer format, and thus, dictate the 
branches to Interactive segments of interest to the 
viewer. 

[0088] Alternatively, the interactive cable box 25 
can be programmed to create a user profile of each 
viewer based on the selections made during one of the 
interactive programs. If the profile shows that a particu- 
lar viewer does not enjoy violence, the system can auto- 
matically branch to another video signal at the 
commencement of a fight in a sports program. Further- 
more, such a user profile could be modified or enriched 
over time based on selections made during future Inter- 
active programs. For example, the 'memory' technique 
described above can be used to modify the user profile 
based on user response over time. 
[0089] Event data Is collected from the viewer when 
the following command is received and processed by 
the controller 260 in the interactive digital cable box 25: 



12 



23 



EP 1 021 037 A2 



24 



VIEWER PROFILE 

This command is used to enable events which 
can be utilized for profiling during a show. The 
events indicated in this command are those profil- 
ing events which are possible during the show. The 5 
enabled events may be either selected by the 
viewer during the show, or may be automatically 
enabled based upon viewer selections. 

VIEWER_PROFILE EVENT1 ,...EVENT24 io 

[0090] Once the profile is created, the programming 
choices or interactive responses can be triggered based 
on the content of the viewer profile itself. For example, if 
the viewer profile suggests that the viewer is particularly is 
interested in sports cars, a sports car commercial could 
be played for the viewer at a predetermined point in the 
program. As another application, if a viewer's profile 
indicates that the viewer is interested in cooking, when- 
ever the viewer watches such a program, the user pro- 20 
file would trigger the interactive program to download 
recipes and either display such recipes on the screen or 
send the recipes to an attached printer. 
[0091] Viewer profile information can then be col- 
lected at the control studio 5 through polling of the 25 
viewer cable boxes 25 for viewer selection data. Special 
polling software is loaded into the digital set top box 25 
for performing the polling functions. Alternatively, the 
invention allows for the digital set top boxes 25 to send 
back data on command from the control studio 5 or peri- 30 
odicalty. The command to initiate an upload of viewer 
profile data is as follows: 

UPLOAD EXTENDED 

This command is used to Initiate the uploading 35 
of data to the central site. 

UPLOAD.EXTENDED UPLOADJDENTIFIER 
VARIABLE.NAME [PHONE.NUMBER] 

40 

UPLOADJDENTIFIERnumeric constant identifying 
upload. 

VARIABLE_NAME name of variable to be 
uploaded 

PHONE_NUMBER string, that consists of num- 45 
bers only, it can be omitted, if 
not needed. 

[0092] Regardless of whether a polling or periodic 
scheme is used, the statistics and other user profile so 
information is preferably sent back to the control studio 
5 by use of the back-channel. 

H. Applications 

55 

[0093] The embodiments, described above, allow 
for several possible applications. For example, in a live 
sports event 10, one channel could carry the standard 



video channel, with other channels carrying different 
camera angles 100 and/or closeups of particular play- 
ers. Other potential video options include instant replay, 
highlights, player statistics via graphic overlays, etc, 
Graphics presenting statistical information on the play- 
ers can be constantly updated using the Chyron sys- 
tem. 

[0094] The provision of trigger points 500, 
explained above, can provide for seamless integration 
of such video options during the sporting event, based 
on either the viewer responses to inten-ogatories at the 
beginning of the program and/or on the digital viewer 
profile. 

[0095] Further, the viewer can become the director 
with the present invention. For example, the viewer can 
choose which camera angles 100 to emphasize. In a 
broadcast of golf, the viewer can direct whether they 
desire to follow a particular player from hole-to-hole, 
focus on one particularly difficult hole. In this manner, 
the viewer can customize the sporting broadcast to 
meet his own interests. 

[0096] Further, the viewer can act as a director to 
create their own video. During a live concert 10, for 
example, the viewer selects various camera angles 100 
at different times. These selections along with a time 
stamp, indicating the program time when each selection 
was made, are stored in memory 265. When the pro- 
gram is played back a second time, the processor 260 
will automatically direct branching between the video 
channels according to the stored selections at the time 
stamp. In this manner, a music video can be created by 
the viewer. 

[0097] The viewing experience can be further 
enlightening for the viewer by implementing games and 
contests during the live sporting event 10. For example, 
graphics overlays can be developed that query the 
viewer during the game. During a football broadcast, for 
example, viewers can be queried with such interrogato- 
ries as the following: 

What will be the next play? (RUN/PASS/KICK); 

Will the offense get the first down?; 

Will they score on this possession?; 

Pick the halftime score; 

Who will win? 

[0098] Each viewer's responses can be sent back 
to the control studio 5 for tabulation of scores. Prefera- 
bly, the responses are packaged at the digital cable box 
25 and transmitted to the control studio 5 via the digital 
backchannel upon the UPLOAD EXTENDED com- 
mand. Alternatively, tabulation of scores can take place 
at the digital cable box 25 through the utilization of cer- 
tain software in memory 265. Each correct answer can 



13 



25 



EP 1 021 037 A2 



26 



correspond to a certain number of points. At the end of 
the game, the interactive program preferably presents a 
graphic showing the viewer point total. If desired, adver- 
tisers could present special gift certificates for excellent 
performance in such games. The provision of such cer- 
tificates would occur by displaying a certain code that a 
viewer can take to a store to receive the gift. In this man- 
ner, viewer interests in sports events can be enhanced. 
[0099] Further, the viewer has the option with the 
present invention to block out viewing of certain events. 
For example, if the viewer is adverse to violence during 
a sporting event or other type program, the system can 
block out such options from the viewer with the following 
data commands: 

EVENT 

This command is used to indicate occurrence 
of a certain event (e.g., a fight breaking out during a 
football game). 

EVENT EVENT.NUMBER 

EVENT_NUMBER is a numeric constant. 

ON EVENT EXECUTE MACRO 

This command is used to implement certain 
actions (such as an automatic branch to another 
video channel in order to block out violent event, for 
example), as soon as the event arrives. 

ON_EVENT EVENT.NUMBER 
MACRO.NUMBER 

EVENT_NUMBER is a numeric constant. 
MACRO_NUMBER is a numeric constant. 
I. Two- Way Configuration 

[0100] The live programming system of the present 
invention may be operated in a two way configuration, 
as illustrated in Figure 7. In this mode, the various video 
signals are processed as previously described, being 
digitized and compressed at the control studio 5. The 
signals are then sent to a central switching station, or 
headend 30. 

[0101] In this embodiment, the switching between 
the various live digital signals is accomplished at the 
headend 30 rather than at the receiver. On the receive 
end, each digital set-top box 760 relays viewer selec- 
tions back to the remotely located switching station 30. 
Preferably, the viewer selections are relayed by way of 
the digital back channel 770. However, the viewer selec- 
tions may be relayed to the switching station 30 by any 
conventional means, such as two-way cable television, 
telephone or microwave transmission. The switching 
station 30 receives the viewer selection and routes the 
desired signal to a transmitter 750 which conventionally 



transmits the desired video down the appropriate digital 
cable channel for the particular viewer. 
[0102] At the central switching station 30, a demul- 
tiplexer 710 demultiplexes the compressed signals and 

5 places each on a separate bus channel 725. A number 
of remote control interactive switches 730, 732, 734, 
736 are connected to the video signal bus 725. Based 
on the viewer selections, an algorithm stored in memory 
265 and under processor 260 control at the central 

10 switching station 30, a digital seamless switch is made 
and the selected video, audio and/or graphics are for- 
warded to the viewer home for display 
[0103] Such a two-way embodiment could be imple- 
mented in a video dial tone or video server system. In 

15 such a system, only a single video channel 755 is nec- 
essary for each home. Once the viewer selection is 
received at the server site at the cable headend 30, a 
switch is made to the appropriate video stream and this 
stream is sent on the single channel 755 to the home. 

20 [0104] Alternatively, it may be desirable to transmit 
an interactive sporting event over a single telephone 
line. When the viewer enters a selection on their remote 
20, a signal is sent by way of the telephone line to the 
central switching station 30 which routes the desired 

25 signal of the interactive program over the user's tele- 
phone line so that a single link handles both the interac- 
tive choice being made at the receiver and the 
transmission of that choice from the headend 30 where 
the actual switching takes place in response to the inter- 

30 active selection made at the receiver. 

[0105] The two-way link between the viewer and 
switching station 30 may be used for other purposes. 
For example, demographic data may be transferred 
from the viewer to the broadcast network for commercial 

35 purposes, such as targeted advertising, billing, or other 
commercial or non-commercial purposes. 
[0106] While the present invention has been 
described primarily with respect to live events, and in 
particular sporting events, It has equal potential for 

40 enhancing content in other program categories. A 
viewer can become their own director of a murder mys- 
tery or other drama. By entering responses to displayed 
questions at the initiation of or during the show, the pro- 
gram will branch to alternative video/audio segments as 

45 a result of the user selections. In this manner, different 
viewers with different selections may end up with a dif- 
ferent murderer at the conclusion of the broadcast. 
[0107] Using the foregoing embodiments, methods 
and processes, the interactive multimedia computer 

50 maximizes personalized attention and interactivity to 
subscribers in their homes in real time. Although the 
present invention has been described in detail with 
respect to certain embodiments and examples, varia- 
tions and modifications exist which are within the scope 

55 of the present invention as defined in the following 
claims. 
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Claims 

1. A method for providing live interactive digital pro- 
gramming, comprising: 

receiving live Interactive programming, the live 
Interactive programming comprising a plurality 
of digitally compressed video and audio, and 
one or more information segment addresses 
specifying one or more addresses of related 
information segments, the reception system 
comprising: 

obtaining viewer entries; 

selecting one of the video and audio signals 
and directing a switch to the selected video and 
audio signals; 

decoding the information segment addresses; 

retrieving the one or more information seg- 
ments residing at the determined addresses; 

demultiplexing the selected video and audio 
signals; 

decompressing the demultiplexed selected 
video and audio signals; and 

presenting the video, audio signals and infor- 
mation segments. 



obtaining at least one address associated with 
at least one Web site; and 

automatically establishing, based on a user 
5 response, a communications link with the 

address indicated Web site. 

4. A method a claimed in Claim 3, further comprising 
the step of using the communications link to receive 

10 information from the Web site, 

5. A method of providing to a user digital interactive 
programming at a receiver station, and during view- 
ing of one of a plurality of video signals associated 

15 with the digital programming, accessing an Internet 
site through an Internet address received at the sta- 
tion, comprising the steps of: 

receiving a plurality of digitally compressed 
20 video signals, each signal corresponding to a 

different video option of a program, wherein the 
plurality of video signals comprises a standard 
video signal; 

25 selecting one of the video options; 

digitally decompressing the selected video sig- 
nal corresponding to the selected video option; 
and 

30 

displaying the selected video signal corre- 
sponding to the selected video option; 



2. A method as claimed in Claim 1 , wherein the infor- 
mation segment addresses are uniform resource 35 
locators, the uniform resource locators specifying 
Internet Web sites. 

3. A method of providing digital programming at a 
receiver station, and during display of one of a plu- 40 
rality of video signals associated with the digital 
programming, accessing an Internet site through an 
Internet address, comprising the steps of: 

receiving a plurality of digitally compressed 45 
video signals, each signal corresponding to a 
different video option of a program, wherein the 
plurality of video signals comprises a standard 
video signal; 

50 

selecting one of the video options; 

digitally decompressing the selected video sig- 
nal corresponding to the selected video option; 
and 55 

displaying the selected video signal corre- 
sponding to the selected video option; 



obtaining at least one address associated with 
at least one Internet site; 

sending a message request to the address 
indicated Internet site; 

automatically establishing a communications 
link with the address indicated Internet site; 

receiving Internet information from the Internet 
site; 

wherein the communications link is automati- 
cally established with the Internet site without 
user interaction. 

6. A method as claimed in Claim 4 or Claim 5, wherein 
the received information contains at least one 
graphics segment. 

7. A method as claimed in Claim 6, wherein the graph- 
ics segment is displayed to the user. 

8. A method as claimed in any of Claims 4 to 7, 
wherein the received information contains a video 
segment. 
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9. A method as claimed in Claim 8 wherein the video 
segment is displayed to the user. 

10. A method as claimed in any of Claims 4 to 9, 
wherein the received information contains an audio 5 
segment. 

11. A method as claimed in Claim 10, wherein the 
audio segment is played for the user. 

10 

12. A method as claimed in any of Claims 3 to 11, 
wherein the receiver station is a digital television. 

13. A method as claimed in any of Claims 3 to 11, 
wherein the receiver station comprises a personal is 
computer with a television card. 

14. A method as claimed in any of Claims 3 to 11, 
wherein the receiver station comprises a digital 
cable box and a television, operably connected to 20 
the digital cable box. 

15. A method as claimed in any of Claims 3 to 14, 
wherein the plurality of video signals further com- 
prise at least one closeup video and at least one 25 
slow motion video replay. 

16. A method as claimed in any of Claims 3 to 14, 
wherein the selected video option corresponds to 
the replay video. 30 

17. A method as claimed in any of Claims 3 to 16, 
wherein the digital programming is received from a 
digital versatile disk. 

35 

18. A method as claimed in any of Claims 3 to 16, 
wherein the digital programming is received from a 
CD ROM. 

19. A method as claimed in any of Claims 3 to 16, 40 
wherein the digital programming is received from an 
Internet connection. 

20. A method as claimed in any of Claims 3 to 16, 
wherein the digital programming is received from 45 
cable headend. 

21. A method as claimed In any of Claims 3 to 16, 
wherein the digital programming is received from 
satellite broadcast. so 

22. A method as claimed in any of Claims 3 to 21 , fur- 
ther comprising the steps of: 



30 

23. A method as claimed in any of Claims 3 to 22, fur- 
ther comprising the steps of: 

obtaining a plurality of graphics segments; 
selecting at least one graphic segment; 
displaying the selected graphic segments. 

24. A method as claimed in Claim 23, wherein at least 
one of the graphics segments is stored locally to the 
receiver station. 

25. A method as claimed in Claim 23, wherein at least 
one of the graphics segments is received over a 
communications link from a remote source. 

26. A method as claimed in Claim 23, wherein at least 
one of the graphics segments is received over a 
communications link with an online service pro- 
vider. 

27. A method as claimed in any of Claims 3 to 26, fur- 
ther comprising the step of receiving a plurality of 
audio signals. 

28. A method as claimed in Claim 27, wherein each 
audio signal is associated with one of the video sig- 
nals. 

29. A method as claimed in Claim 27, wherein at least 
one of the audio signals is stored locally to the 
receiver station. 

30. A method as claimed in Claim 27, wherein at least 
one of the audio signals is received over a commu- 
nications link from a remote source. 

31. A method as claimed in Claim 27, wherein at least 
one of the audio signals is received over a commu- 
nications link with an online service provider. 

32. A system for providing digital programming at a 
receiver station, and during display of one of a plu- 
rality of video signals associated with the digital 
programming, accessing an Internet site through an 
Internet address, comprising: 

means for receiving a plurality of digitally com- 
pressed video signals, each signal correspond- 
ing to a different video option of a program, 
wherein the plurality of video signals comprises 
a standard video signal; 

processing means, connected to the receiving 
means, the processing means selecting one of 
the video options; 



creating a viewer profile; 55 
wherein the selecting step comprises selecting 
the video option based at least in part on the 
viewer profile. 
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a digital decompressor, operably connected to 
the processing means, for decompressing the 
selected video signal corresponding to the 
selected video option; and 

a display, operably connected to the digital 
decompressor, for displaying the selected 
video signal corresponding to the selected 
video option; 

means for obtaining at least one address asso- 
ciated with at least one Web site; and 

means for automatically establishing, based on 
a user response, a communications link with 
the address indicated Web site. 

33. A system as claimed in Claim 32, further compris- 
ing means for receiving information from the Web 
site. 

34. A system for providing digital programming at a 
receiver station, and during display of one of a plu- 
rality of video signals associated with the digital 
programming, accessing an Internet site through an 
Internet address received at the station, compris- 
ing: 

means for receiving a plurality of digitally com- 
pressed video signals, each signal correspond- 
ing to a different video option of a program, 
wherein the plurality of video signals comprises 
a standard video signal; 

processing means, connected to the receiving 
means, the processing means selecting one of 
the video options; 

a digital decompressor, operably connected to 
the processing means, for decompressing the 
selected video signal corresponding to the 
selected video option; and 

a display operably connected to the digital 
decompressor, for displaying the selected 
video signal corresponding to the selected 
video option; 

means for obtaining at least one address asso- 
ciated with at least one Internet site; 

means for automatically establishing a commu- 
nications link with the address indicated Inter- 
net site; and 

means for receiving Internet information from 
the Internet site; 

wherein the communications link Is automati- 



cally established with the Internet site without 
user interaction. 

35. A system as claimed in Claim 33 or Claim 34, 
5 wherein the received information contains at least 

one graphics segment. 

36. A system as claimed in Claim 35, wherein the 
graphics segment is displayed on the display 

10 screen. 

37. A system as claimed in Claim 33 or 34, wherein the 
received information contains a video segment. 

15 38. A system as claimed in Claim 37, wherein the video 
segment is displayed on the display screen. 

39. A system as claimed in Claim 33 or 34, wherein the 
received information contains an audio segment. 

20 

40. A system as claimed in Claim 39, further compris- 
ing a speaker, wherein the audio segment is played 
on the speaker. 

25 41. A system as claimed in any of Claims 32 to 40, 
wherein the receiver station comprises a digital tel- 
evision. 

42. A system as claimed in any of Claims 32 to 40, 
30 wherein the receiver station comprises a personal 

computer with a television card. 

43. A system as claimed in any of Claims 32 to 40, 
wherein the receiver station comprises a digital 

35 cable box and a television, operably connected to 
the digital cable box. 

44. A system as claimed in any of Claims 32 to 43. 
wherein the plurality of video signals further com- 

40 prise at least one closeup video and at least one 
stow motion video replay. 

45. A system as claimed in any of Claims 32 to 43, 
wherein the selected video option corresponds to 

45 the replay video. 

46. A system as claimed in any of Claims 32 to 45, fur- 
ther comprising a digital versatile disk, wherein the 
digital programming is received from the digital ver- 

50 satile disk. 

47. A system as claimed in any of Claims 32 to 45, fur- 
ther comprising a CD ROM, wherein the digital pro- 
gramming is received from the CD ROM. 

55 

AS, A system as claimed in any of Claims 32 to 45, 
wherein the digital programming is received from an 
Internet connection. 
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49. A system as claimed in any of Claims 32 to 45, 
wherein the digital programming Is received from a 
cable headend. 

50. A system as claimed in any of Claims 32 to 45, 5 
wherein the digital programming is received from 
satellite broadcast. 

51 . A system as claimed in any of Claims 32 to 50, fur- 
ther comprising a storage device, wherein a viewer io 
profile Is stored in the storage device and the proc- 
essor selects the video option based at least in part 

on the viewer profile. 

52. A system as claimed in any of Claims 32 to 51 , fur- is 
ther comprising: 

means for obtaining a plurality of graphics seg- 
ments; 

wherein the processing means selects at least 20 
one graphic segment and the selected graph- 
ics segment is displayed on the display. 

53. A system as claimed in Claim 52, further compris- 
ing a storage device, wherein at least one of the 25 
graphics segments is stored in the storage device. 

54. A system as claimed in Claim 52, wherein at least 
one of the graphics segments is received over a 
communications link from a remote source. 30 

55. A system as claimed in Claim 52, wherein at least 
one of the graphics segments is received over a 
communications link with an online service pro- 
vider. 35 

56. A system as claimed in any of Claims 32 to 55, fur- 
ther comprising means for receiving a plurality of 
audio signals. 

40 

57. A system as claimed in Claim 56, wherein each 
audio signal is associated with one of the video sig- 
nals. 

58. A system as claimed in Claim 56 or Claim 57, fur- 45 
ther comprising a storage device, wherein at least 
one of the audio signals is stored in the storage 
device. 

59. A system as claimed in Claim 56 or Claim 57, so 
wherein at least one of the audio signals Is received 
over a communications link from a remote source. 

60. A system as claimed In Claim 56 or Claim 57, 
wherein at least one of the audio signals is received 55 
over a communications link with an online service 
provider. 
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